Return POST /get_setid results as a JSON list, as NodeNorm Redis does - #48

…document (#153)

NodeNorm Redis and NodeNorm ES are meant to be interchangeable for the
endpoints they share, but three NodeNorm ES endpoints answered
differently from Redis and from ES's own OpenAPI document: `GET
/get_allowed_conflations` returned HTTP 500, `POST /get_setid` returns
an object keyed `"0"`, `"1"`, … instead of a list, and
`/get_semantic_types` lower-cases Biolink classes. This PR adds
`tests/nodenorm/test_nodenorm_endpoints.py`, which runs against every
target and would have caught all three. It also checks that each
deployment documents the endpoints both backends share, and that every
path its OpenAPI document lists is actually served.

biothings/NodeNormalizationAPI#58, released as NodeNorm ES 1.2.0, fixed
the conflations 500; `ci` and `test` run 1.2.0, prod still runs 1.0.0.
The setid shape and the semantic-types casing are fixed by
biothings/NodeNormalizationAPI#48 and #59, both still open. What is
still wrong is a strict xfail on the ES targets, so an unexpected pass
reports when a marker can come out.

## Calls worth overruling

- **Each xfail passes `raises=KnownElasticsearchDiscrepancy`, and that
class is raised only on the bug's own signature.** That means only
lower-cased `biolink:` classes, only an object keyed `"0"`..`"n-1"` with
a set ID under each key, and only Tornado's bare 500 page while
`get_normalized_nodes` still works. Anything else the same check catches
fails with a plain assert. A bare strict xfail, or a class raised for
"not a list" or "any 500", would also absorb an outage or an unrelated
regression.
[`CLAUDE.md`](https://github.com/TranslatorSRI/babel-validation/blob/46e69ea/CLAUDE.md#testing-patterns)
records the pattern, and why an imperative `pytest.xfail()` doesn't work
with the dashboard.
- **The conflations xfail is keyed on the version `/status` reports**,
not just the backend: ES targets older than 1.2.0 expect the 500, later
ones must pass. `nodenorm_version()` in `service_checks.py` reads
`1.2.0`, `v1.2.1`, `1.3`, `1.3.0-rc1` and PEP 440's `1.3.0.dev0` or
`1.2.0.post1`; a suffix is ignored, so a pre-release counts as its
release. A target whose `/status` has no release number at all fails
rather than guessing.
- **The path sweep skips `/get_allowed_conflations` where its 500 is
expected**, instead of being a second xfail for the same bug.
`test_get_allowed_conflations` tracks it, and the sweep still checks
every other path on prod.
- **The shared-endpoint list is the five both backends serve**, not
everything Redis documents. NodeNorm ES deliberately leaves out
`/query`, `/asyncquery` and `/get_curie_prefixes`
(biothings/NodeNormalizationAPI#36).
- **A path counts as served unless a bare GET returns 404 or 5xx.** A
POST-only or parameter-requiring path legitimately answers 405, 400 or
422. A timeout or refused connection is recorded against its path rather
than raised, so one hung path doesn't hide the rest.
- **The OpenAPI document is treated as untrusted.** Its path keys must
be plain `/segment` paths, and a document may list at most 20, before
any URL is built from them, so a key such as `//host/x` cannot send a
request elsewhere. The sweep requests five paths at a time with a (5s,
10s) connect/read timeout, so even a hung deployment fails inside
pytest-timeout's 300s with each path named. Every message goes through
`truncated_repr()`.
- **The setid test checks only the shape**, with a single POST.
`test_setid_basic` already compares the set IDs from GET and POST. It
now asserts the POST succeeded and returned a list, showing the status
and body, rather than failing with a `TypeError`. It gets no xfail,
because it also fails on Redis targets today (water now normalizes to
`CHEBI:15377`), and an xfail would hide that.

## Also here

- `service_checks.py` gains `get_openapi()`, which caches each target's
OpenAPI document per process the way `get_status()` caches `/status`.
Both `test_openapi_json` tests and the new tests share it.
- The shared fetch checks for a JSON object inside the cache, so a brief
200 with `null` is fetched again by the next test rather than
remembered.
- Every request in `test_nodenorm_setid.py` now has a timeout.

## Verification

On 46e69ea, against `prod`, `test`, `ci`, `ci-nameres-es`, `dev` and
`exp`, every test in the new file passes or xfails as expected. The nine
xfails are semantic types and setid on the four ES targets, plus
conflations on prod. The unit suite passes. Outside the new file, the
touched tests fail on exactly the targets they fail on `main`, for
reasons in the deployments: missing or wrong `x-translator` blocks and a
`maxEdges` schema error in `test_openapi_json`, non-JSON setid bodies on
the ES targets, and water now normalizing to `CHEBI:15377` in
`test_setid_basic`. Faked responses confirmed that an error object, a
non-Biolink type, or a 500 during an outage or on 1.2.0 fails outright.

## After merging

Nothing blocks merging. The markers come out as the strict XPASSes
report: conflations once prod reaches 1.2.0, setid with
biothings/NodeNormalizationAPI#48, and semantic types with
biothings/NodeNormalizationAPI#59. Whether lower-cased Biolink classes
would also be acceptable is a separate question, #158.

🤖 Generated with [Claude Code](https://claude.com/claude-code)